[Flang][OpenMP] Move builtin .mod generation into runtimes #137828

Meinersbur · 2025-04-29T15:47:32Z

Move building the .mod files from openmp/flang to openmp/flang-rt using a shared mechanism. Motivations to do so are:

Most modules are target-dependent and need to be re-compiled for each target separate, which is something the LLVM_ENABLE_RUNTIMES system already does. Prime example is iso_c_binding.mod which encodes the target's ABI. Most other modules have #ifdef-enclosed code as well.
CMake has support for Fortran that we should use. Among other things, it automatically determines module dependencies so there is no need to hardcode them manually.
It allows using Fortran itself to implement Flang-RT. Currently, only iso_fortran_env_impl.f90 emits object files that are needed by Fortran applications ([flang] Linker for non-constant accesses to kind arrays (integer_kind, logical_kind, real_kind) #89403). The workaround of [flang][runtime] Build ISO_FORTRAN_ENV to export kind arrays as linkable symbols #95388 could be reverted.

Some new dependencies come into play:

openmp depends on flang-rt for building lib_omp.mod and lib_omp_kinds.mod. Currently, if flang-rt is not found then the modules are not built
check-flang depends on flang-rt: If not found, the majority of tests are disabled. If not building in a bootstrpping build, the location of the module files can be pointed to using -DFLANG_INTRINSIC_MODULES_DIR=<path>, e.g. in a flang-standalone build. Alternatively, the test needing any of the instrinsic modules could be marked with REQUIRES: flangrt-modules which would affect ~217 files.
check-flang depends on openmp: Not a change; tests requiring lib_omp.mod and lib_omp_kinds.mod those are already marked with openmp_runtime.

As intrinsic are now specific to the target, their location os moved from include/flang to <resource-dir>/finclude/<triple>. The mechnism to compute the location have been moved from flang-rt (previously used to compute the location of libflang_rt.*.a) to common locations in cmake/GetToolchainDirs.cmake and runtimes/CMakeLists.txt so they can be used by both, openmp and flang-rt. Potentially the mechnism could also be shared by other libraries such as compiler-rt.

finclude was chosen because gfortran uses it as well and avoids misuse such as #include <flang/iso_c_binding.mod>. The search location is now determined by ToolChain in the driver, instead of by the frontend. Now the driver adds -fintrinsic-module-path for that location to the frontend call (Just like gcc does). -fintrinsic-module-path had to be fixed for this because ironically it was only added to searchDirectories, but not intrinsicModuleDirectories_. Since the driver determines the location, tests invoking flang -fc1 and bbc must also be passed the location by llvm-lit. This works like llvm-lit does for finding the include dirs for Clang using -print-file-name=....

Related PRs:

[Flang] Solved issue with inline compiler directive #143699 (allows removing the workaround in flang-rt/lib/runtime/ieee_arithmetic.f90)
[Flang] Add -fintrinsic-modules-path= alias #149417 (allows removing the SHELL: workaround)
[OpenMP] Fix runtimes default build #149871 (fixes a build configuration issue)

jhuber6

What's the main limitation here? If this is just a file dependency it should be identical to how all the OpenMP tests depend on omp.h being in the resource directory. IMHO this is trivial if we do a runtimes build, since we can just require that openmp;flang-rt are in the same toolchain, which then gives us well defined access to openmp's CMake targets so long as it's listed before flang-rt.

clang/include/clang/Driver/Options.td

clang/include/clang/Driver/ToolChain.h

clang/lib/Driver/ToolChain.cpp

flang-rt/CMakeLists.txt

Meinersbur · 2025-04-29T16:20:22Z

While I appreciate the review, it is not yet in the state that warants one. It is still in an experimentation stage, so I did not yet care about formatting. There are also a lot of changes in here that will eventually not be needed.

Goals are:

Currently modules files are expected at $prefix/include/flang/*.mod where prefix is the parent of bin where flang is located. It should be in $prefix/lib/clang/finclude/<triple>/*.h, i.e. the resource directory since mod-files are specific to the version of flang, and distinct for each target triple since the mod files can be different for each target. Necessary for cross-compilation. In addition to the CMake code, the flang driver code needs to change as well because it hardcodes $path/../include/flang.
Use cmake to build the module files within the flang-rt/ runtime. The LLVM_ENABLE_RUNTIMES system handles which target triples to build and Flang being available. CMake should care about the build dependencies. Have to change the driver again to tell it where to emit the module files.
Use the same mechanism as above to build omp_lib.mod and omp_lib_kinds.mod, but in the openmp/ runtime. Since in the same CMake builddir, CMake will ensure dependencies already.
This means flang-rt (and openmp) is necessary to be compiled before running flang's tests which require those module files. Flang's OpenMP tests already require openmp's modules to be compiled, it will be no different to flang-rt's builtin modules.

Sounds relatively simple, but there have been many small issues, starting with CMake's misspelling of CMAKE_Fortran_BUILDING_INSTRINSIC_MODULES.

DanielCChen · 2025-05-07T15:05:53Z

It should be in $prefix/lib/clang/finclude/<triple>/*.h

Just want to make sure: Should it be $prefix/lib/clang/${LLVM_VERSION_MAJOR}/finclude/<triple>/*.mod?

Meinersbur · 2025-05-07T20:35:51Z

Just want to make sure: Should it be $prefix/lib/clang/${LLVM_VERSION_MAJOR}/finclude/<triple>/*.mod?

That is correct, I forgot the version number that is part of the resource directory.

flang-rt/lib/runtime/CMakeLists.txt

github-actions · 2025-07-17T13:07:13Z

✅ With the latest revision this PR passed the Python code formatter.

Meinersbur · 2025-07-17T13:24:25Z

What's the main limitation here? If this is just a file dependency it should be identical to how all the OpenMP tests depend on omp.h being in the resource directory.

omp.h is created by configure_file at configure time. No dependency other than runtimes-configure needed.

IMHO this is trivial if we do a runtimes build, since we can just require that openmp;flang-rt are in the same toolchain,

With toolchain you mean a bootstrapping build with LLVM_ENABLE_RUNTIMES=openmp;flang-rt ? Don't forget the users of a Flang-standalone build (cmake -S <llvm-project>/flang).

which then gives us well defined access to openmp's CMake targets so long as it's listed before flang-rt.

check-flang (LLVM_ENABLE_PROJECTS=flang) needs access to libomp.mod (LLVM_ENABLE_RUNTIMES=openmp) and the flang modules as well to work.

The default build of openmp (`cmake -S <llvm-project>/runtimes -DLLVM_ENABLE_RUNTIMES=openmp`) current fails with ``` CMake Error at /home/meinersbur/src/llvm/flangrt/_src/cmake/Modules/GetClangResourceDir.cmake:17 (string): string sub-command REGEX, mode MATCH needs at least 5 arguments total to command. Call Stack (most recent call first): /home/meinersbur/src/llvm/flangrt/_src/openmp/CMakeLists.txt:126 (get_clang_resource_dir) ``` The reason is that because it is not a bootstrapping-build, the clang resource dir that it intends to write files such as `omp-tools.h` into, is unavailable. Using the Clang resource dir for writing files is conceptually broken, as that dir might be located in `/usr/lib/clang/<version>/`. Writing to it is only intended in bootstrapping builds where Clang is built alongside openmp. This patch unifies the identification of being in a bootstrapping built. The same `LLVM_TREE_AVAILABLE` definition is going to be used in #137828. No reason for each runtime to define its own.

The default build of openmp (`cmake -S <llvm-project>/runtimes -DLLVM_ENABLE_RUNTIMES=openmp`) current fails with ``` CMake Error at /home/meinersbur/src/llvm/flangrt/_src/cmake/Modules/GetClangResourceDir.cmake:17 (string): string sub-command REGEX, mode MATCH needs at least 5 arguments total to command. Call Stack (most recent call first): /home/meinersbur/src/llvm/flangrt/_src/openmp/CMakeLists.txt:126 (get_clang_resource_dir) ``` The reason is that because it is not a bootstrapping-build, the clang resource dir that it intends to write files such as `omp-tools.h` into, is unavailable. Using the Clang resource dir for writing files is conceptually broken, as that dir might be located in `/usr/lib/clang/<version>/`. Writing to it is only intended in bootstrapping builds where Clang is built alongside openmp. This patch unifies the identification of being in a bootstrapping built. The same `LLVM_TREE_AVAILABLE` definition is going to be used in llvm#137828. No reason for each runtime to define its own.

Meinersbur · 2025-08-04T14:41:05Z

ping

Meinersbur · 2025-08-06T10:42:26Z

Should we choose a different module path than finclude to avoid accidental confusion between flang and gfortran modules files, e.g. fmod?

DavidTruby · 2025-10-21T13:08:09Z

flang-rt/lib/runtime/ieee_arithmetic.f90

+elemental logical function ieee_is_finite_a2(x); real(2), intent(in) :: x;
+!dir$ ignore_tkr(d) x;
+end function ieee_is_finite_a2;
+elemental logical function ieee_is_finite_a3(x); real(3), intent(in) :: x;
+!dir$ ignore_tkr(d) x;
+end function ieee_is_finite_a3;
+ elemental logical function ieee_is_finite_a4(x); real(4), intent(in) :: x;
+!dir$ ignore_tkr(d) x;
+end function ieee_is_finite_a4;
+elemental logical function ieee_is_finite_a8(x); real(8), intent(in) :: x;
+!dir$ ignore_tkr(d) x;
+end function ieee_is_finite_a8;
+elemental logical function ieee_is_finite_a10(x); real(10), intent(in) :: x;
+!dir$ ignore_tkr(d) x;
+end function ieee_is_finite_a10;
+#if FLANG_SUPPORT_R16
+elemental logical function ieee_is_finite_a16(x); real(16), intent(in) :: x;
+!dir$ ignore_tkr(d) x;
+end function ieee_is_finite_a16;
+#endif


Is this change related to the move of the .mod files? It looks like a separate issue to me at a glance

Workaround for #139297. Will not be necessary after landing #143699. See summary.

DavidTruby · 2025-10-21T13:15:22Z

runtimes/CMakeLists.txt

+
+# CMake 3.24 is the first version of CMake that directly recognizes Flang.
+# LLVM's requirement is only CMake 3.20, teach CMake 3.20-3.23 how to use Flang, if used.
+if (CMAKE_VERSION VERSION_LESS "3.24")


The flang support in CMake wasn't really complete/fully functional until 3.28 so I think this should be bumped up to that. Unless you've specifically tested with 3.24 and it doesn't hit the edge cases we didn't fix until later?

I think I tests 3.24 already. I boiled down to CMAKE_Fortran_PREPROCESS_SOURCE not being defined. I will test again before landing.

DavidTruby

This LGTM other than my concerns about the specific CMake version where flang was properly supported. Thanks!

…uiltin-mods

…iltin-mods

…uiltin-mods

…iltin-mods

…uiltin-mods

…iltin-mods

…uiltin-mods

…iltin-mods

…uiltin-mods

…iltin-mods

…uiltin-mods

…iltin-mods

Meinersbur mentioned this pull request Apr 29, 2025

[Flang-rt] Forward libomp-mod dependency for non-default runtime builds #137035

Open

jhuber6 reviewed Apr 29, 2025

View reviewed changes

clang/include/clang/Driver/Options.td Outdated Show resolved Hide resolved

clang/include/clang/Driver/ToolChain.h Outdated Show resolved Hide resolved

clang/lib/Driver/ToolChain.cpp Outdated Show resolved Hide resolved

flang-rt/CMakeLists.txt Outdated Show resolved Hide resolved

DanielCChen reviewed May 7, 2025

View reviewed changes

flang-rt/lib/runtime/CMakeLists.txt Outdated Show resolved Hide resolved

[Flang][Flang-RT][OpenMP] Move builtin .mod generation into runtimes

839198d

Meinersbur force-pushed the flang_builtin-mods branch from 907d3d5 to 839198d Compare July 17, 2025 13:04

python format

a1533cc

Meinersbur requested a review from kkwli July 17, 2025 13:25

Meinersbur added 10 commits July 17, 2025 16:58

Minimize changes in .f90 files

60ad12f

Move CMake 3.20 workaround to where it is needed

e58c524

Don't forget to enable Fortran

7d1a8be

enable_fortran test for CMake 3.20

adc0149

CI test

498d60c

Enforce REAL(16) support

1bb0144

Also add dependency barrier for WIN32

8ec6d03

Dependency barrier test

580a0c5

Dependency barrier info

979691a

Merge branch 'main' into flang_builtin-mods

a720b31

Meinersbur marked this pull request as ready for review July 19, 2025 12:32

Meinersbur requested a review from a team as a code owner July 19, 2025 12:32

sscalpone requested review from klausler and vzakhari July 20, 2025 09:25

Meinersbur requested review from clementval, jeanPerier, sscalpone and tblah July 21, 2025 09:52

Meinersbur added 2 commits July 22, 2025 19:49

Enable preprocessor for test

68efc67

python-format

a00485b

Meinersbur requested a review from petrhosek July 25, 2025 13:31

Meinersbur mentioned this pull request Jul 25, 2025

[OpenMP] Allow Fortran tests #150722

Merged

Meinersbur requested a review from ldionne July 26, 2025 13:10

Meinersbur mentioned this pull request Aug 12, 2025

[flang/flang-rt] Add -isysroot flag only to tests really requiring #152914

Merged

Merge branch 'main' into flang_builtin-mods

bfb16f5

DavidTruby reviewed Oct 21, 2025

View reviewed changes

DavidTruby approved these changes Oct 21, 2025

View reviewed changes

luporl mentioned this pull request Oct 21, 2025

[Flang] .mod files built into include/flang/ do not support multiple target architectures #158790

Open

Meinersbur added 15 commits October 23, 2025 16:56

Merge commit 'e6f5aa9e2b69d1e84bd281c2525004800828b30d^' into flang_b…

593baa6

…uiltin-mods

Merge commit 'e6f5aa9e2b69d1e84bd281c2525004800828b30d' into flang_bu…

08255eb

…iltin-mods

Merge commit 'e915d9addeae5431f869d9a10021382871c11169^' into flang_b…

0454c86

…uiltin-mods

Merge commit 'e915d9addeae5431f869d9a10021382871c11169' into flang_bu…

feffa35

…iltin-mods

Merge commit 'f205a2c0a093767144f474fe54bbfa103e1d6431^' into flang_b…

3cb53d7

…uiltin-mods

Merge commit 'f205a2c0a093767144f474fe54bbfa103e1d6431' into flang_bu…

d3b5285

…iltin-mods

Merge commit '3f3a20f654f913f7e251e3bf4bd5a63e73e5571a^' into flang_b…

2cfab17

…uiltin-mods

Merge commit '3f3a20f654f913f7e251e3bf4bd5a63e73e5571a' into flang_bu…

5ee6a1c

…iltin-mods

Merge commit 'c2765b74ed2bb71bebe96c7dd49f69ade714e5a8^' into flang_b…

72cfb12

…uiltin-mods

Merge commit 'c2765b74ed2bb71bebe96c7dd49f69ade714e5a8' into flang_bu…

b4f0763

…iltin-mods

Merge commit '2499fe1ac40f7a50e71bf93c26921f93df2bcc52^' into flang_b…

a2251d9

…uiltin-mods

Merge commit '2499fe1ac40f7a50e71bf93c26921f93df2bcc52' into flang_bu…

7e5b145

…iltin-mods

Post-merge fixes

a2477de

Merge branch 'main' into flang_builtin-mods

8884ed2

Post-merge fixes

731288c

[Flang][OpenMP] Move builtin .mod generation into runtimes #137828

Are you sure you want to change the base?

[Flang][OpenMP] Move builtin .mod generation into runtimes #137828

Conversation

Meinersbur commented Apr 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jhuber6 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Meinersbur commented Apr 29, 2025

Uh oh!

DanielCChen commented May 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Meinersbur commented May 7, 2025

Uh oh!

Uh oh!

github-actions bot commented Jul 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Meinersbur commented Jul 17, 2025

Uh oh!

Meinersbur commented Aug 4, 2025

Uh oh!

Meinersbur commented Aug 6, 2025

Uh oh!

DavidTruby Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Meinersbur Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

DavidTruby Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

Meinersbur Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

DavidTruby left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Meinersbur commented Apr 29, 2025 •

edited

Loading

DanielCChen commented May 7, 2025 •

edited

Loading

github-actions bot commented Jul 17, 2025 •

edited

Loading

DavidTruby Oct 21, 2025 •

edited

Loading